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SECURITY ANALYST PERFORMANCE 
TRACKING AND ANALYSIS SYSTEM AND METHOD 

Field of the Invention 

The invention relates to a system and methods for managing and viewing 
historical security analyst data; for measuring, analyzing, and tracking the historical 
5 performance of security analysts' estimates and buy/sell recommendations; and using 
such performance and other information to automatically produce better predictors of 
future corporate earnings or stock-price performance. 
Background of the Invention 

There are many individuals who analyze financial data and financial 

10 instruments, such as equity and fixed-income securities. At least some of these 
individuals analyze such data in an attempt to predict future economic events. Such 
individuals may include, for example, security analysts and may be known as 
contributors or analysts, among others. The role of the security analyst is generally 
well-known and includes, among other things, the issuance of earnings or other 

15 financial estimates concerning future economic events and recommendations on 
whether investors should buy, sell, or hold financial instruments, such as equity 
securities. Security analyst estimates may include, but are not limited to, quarterly 
and annual earnings estimates for companies, whether or not they are traded on a 
public securities exchange. 
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At least some investors tend to rely on the earnings estimates and 
recommendations issued by security analysts. Usually more than one analyst follows 
a given equity security. Analysts often disagree on their earnings estimates and 
recommendations and, as a result, analysts 1 earnings estimates and recommendations 
5 may sometimes vary. 

A number of financial information services providers (FISPs) gather and 
report analysts' earnings estimates and recommendations. At least some FISPs 
report the high, low, and mean earnings estimates, as well as mean recommendations 
for equity securities (as translated to a FISP's particular scale, for example, one to 
1 0 five). In addition, FISPs may also provide information on what the earnings 

estimates and recommendations were seven and thirty days prior to the most current 
consensus, as well as the differences between the consensus for a single equity 
security and that of the relevant industry. Moreover, for some clients, FISPs provide 
earnings estimates and recommendations on an analysts-by-analyst basis. An 
1 5 advantage of the availability of analyst-level estimates and recommendations is that a 
client can view the components of the mean estimate or recommendation by analyst. 
Various drawbacks exist, however, with these approaches and other known 
techniques. 

For example, prior approaches include a software program that displays all 
20 current estimates. For a particular fiscal period for a particular security the software 
provides the ability to simply "include" or "exclude" each estimate 
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(recommendation) from the mean. This is problematic for several reasons. First, 
commercially available databases of estimates and recommendations contain 
"current" data on thousands of stocks. Each stock may have estimates from 1 to 70 or 
more analysts. Each analyst may provide estimates for 1 to many periods. The data 
may be updated throughout the day. Manually dealing with this volume of 
information can be time consuming. 

A second drawback is that with current techniques, if someone were inclined 
to determine which estimates (recommendations) should get more weight, and 
which estimates should get less or no weight, the sheer volume of analysts (over 
3,000 for U.S. stocks alone) makes it extremely difficult to know which analysts 
provide more useful information than others. Current techniques lack sufficient 
ability to intelligently measure historical analyst performance and beneficially use 
such measurements. 

A third drawback is that it while it is possible to imagine various weighting 
systems or algorithms, it is difficult to effectively implement or test them. Current 
systems do not provide the ability to effectively devise new estimate 
(recommendation) weighting algorithms; nor do they provide the ability to easily test 
their (hypothetical) historical performance. 

A fourth drawback with current techniques is that there are limited or no 
tools for effectively viewing historical estimates and recommendations as time-series 
graphs or for overlaying this information over a graph of prices for the securities to 
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understand the relationship between changes in estimates (recommendations) to 
changes in securities prices. These and other drawbacks exist with existing systems 
Summary of the Invention 
5 An object of the invention is to overcome these and other drawbacks of prior 

approaches and techniques. 

Another object of the invention is to measure and use the historical 
performance of an analyst's past estimates or recommendations to better predict 
future earnings or effectively use analysts' recommendations. 
10 Another object of the invention is to provide a tool to automatically create 

more accurate composite estimates (predictors) by adjusting one or more analyst's 
estimates up (down) if they have a historical tendency to under (over) estimate the 
value of future quantities such as earnings. 

Another object of the invention is to automatically create an improved 
15 composite estimate for one or more securities by calculating a weighting factor for 
each analyst's estimate that gives relatively more weight to certain analyst estimates 
and relatively less or no weight to other analyst estimates, where the weighting factor 
is based upon predetermined criteria. 

Another object of the invention is to objectively measure the historical 
20 accuracy of the estimates made by one or more Contributors (analysts and/or brokers) 
Another object of the invention is to provide a tool to measure historical 
accuracy of predictions with the flexibility for the user to specify one or more of the 
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time frame of estimates to be measured (e.g., those estimates 9-12 months prior to 
actual report date), the number of periods over which to aggregate performance, the 
error metric used to calculate performance, and the stocks over which to aggregate 
performance. 

5 Another object of the invention is to exclude from calculation of an 

automatically generated composite all estimates (recommendations) that do or do not 
meet certain criteria. 

Another object of the invention is to provide a tool to automatically identify 
a cluster, or major revision of estimates (or recommendations), based upon 
10 predetermined criteria 

Another object of the invention is to automatically create a composite 
estimate (or recommendation) by excluding (or assigning reduced weight) to those 
estimates received prior to the beginning of a cluster. 

Another object of the invention is to automatically calculate an improved 
15 composite estimate or recommendation by both adjusting estimates of earnings by 
calculating an earnings estimate based upon the adjustment and weighting factors for 
a plurality of analysts' estimates. 

Another object of the invention is to measure the historical performance of a 
single or plurality of analysts* estimates and to measure the historical profitability of 
20 recommendations of either a single or plurality of analysts. 
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Another object of the invention is to compare the performance of analysts' 
estimates and recommendations for a particular financial instrument or industry. 

Another object of the invention is to enable a user to define the method for 
measuring the performance of an analyst by allowing for the specification of a 
5 number of security analyst performance parameters or metrics. 

Another object of the invention is to allow for the creation, storage, and recall 
of security autoweight models and related estimates where the models automatically 
assign weights to each analyst's estimate or recommendation based on stored, user- 
defined criteria. 

10 Another object of the invention is to test autoweight models by applying 

predetermined criteria. 

Another object of the invention is to provide a data visualization technique to 

allow a user to display simultaneously the estimates of earnings (or other quantities 

such as revenues) for a single or plurality of contributors using predetermined 
15 criteria, along with the actual earnings (or other quantity) corresponding to those 

estimates and other related parameters or metrics. 

Another object of the invention is to provide a data visualization technique to 

allow a user to display simultaneously time-series charts of estimates of earnings (or 

of estimates of other quantities such as revenues, or of recommendations) for a single 
20 or plurality of contributors using predetermined criteria, along with a time-series of 

the security's price over the corresponding time interval. 



6 



WO 99/56192 PCT/US99/08909 



Another object of the invention is to provide a data visualization technique to 
allow one to display, as either raw numerical data, a chart, or graph, a number of 
earnings estimate performance metrics for either a single or plurality of analysts, 
based upon predetermined criteria. 
5 Another object of the invention is to provide a data visualization technique to 

display simultaneously the numerical representation of a single or plurality of 
security analysts' purchase recommendations for predetermined .criteria, along with 
the actual change in the value of the security corresponding to the 
recommendation(s). 

10 These and other objects of the invention are carried out, according to various 

preferred embodiments of the invention. 

According to one embodiment of the invention, an improved estimate of a 
future earnings event can be automatically developed for one or a plurality of 
securities by applying a weighting factor to each analyst's estimate. The weighting 

15 factor may be based on a variety of factors as specified in an autoweight model such 
as the relative recency of each analyst's estimate, the analyst's historical 
performance, or other factors. For example, if an estimate is relatively old, it may get 
a relatively low or zero weighting, whereas more recent estimates may be given a 
relatively high weighting. The sum of the weights assigned to analyst's estimates for 
20 a particular fiscal period should equal one. Two or more factors may be used in 
combination. Autoweight models allow for the definition of an arbitrary number of 
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different factors. For example, an analyst may receive a score for her past 
performance and another score for the recency of her estimate. Using a pre-defined 
function, these factor scores can be consolidated with the result being a summary 
weight for each analyst. Using the automatically calculated weights, the estimates 
5 and revisions of either a single or plurality of analysts' can be composited for a given 
fiscal period by calculating the weighted average of estimates such that an improved 
estimate can be calculated. For example, a custom composite estimate or composite 
estimate may be calculated by multiplying a plurality of analysts' current earnings 
estimates for a particular security by their respective weighting factors and then 

10 summing over each estimate. Similarly, analyst recommendations at a point in time 
can be automatically weighted according to different factors to create an improved 
composite recommendation by multiplying each analyst's recommendation by that 
analyst's weighting factor. 

According to another aspect of the invention, a system and method enable a 

15 user to track, analyze, and compare analysts' past performances. According to this 
embodiment, a database may be provided that contains information about analyst's 
past performance through the combination of each analyst estimate record in the 
database has a number of predetermined data, fields. 

The database may comprise a combination of estimate data comprising raw 

20 data regarding estimates and performance for analysts and a pre-calculated values 
that are calculated and stored in the database for further analysis. The pre-calculated 
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values may be based on the estimate data loaded from a secondary source analyst 
performance database or maintained on that database from a transaction processing 
system, for example. 

These pre-calculated values may include error metric values for each security, 
5 for each historical fiscal period in the database, for each contributor (e.g., 
analyst/broker pair) is a row in the database. Each row comprises multiple error 
metrics valued over a range of time periods. For example, if three error metrics were 
provided, the system may maintain a value for error metric 1 over a 0-3 month time 
period, error metric 1 over a 3-6 month time period, error metric 2 over a 0-3 month 
10 time period, error metric 2 over a 3-6 month time period, error metric 3 over a 0-3 
month time period, and error metric 3 over a 3-6 month time period. Of course, a 
great number of such error metrics may be stored and the number and ranges of the 
time periods may also vary according to the present invention. 

Example fields for the pre-calculated portion of the database may comprise 
15 an analyst identifier, an event identifier, an analyst estimate date, a raw error 
indicator (analyst estimate minus the actual earnings for a particular event), other 
error metrics, such as the error percent to actual earnings, error percent to consensus, 
other user-defined error metrics, and the number of days between the estimate of an 
event and the actual event. This last parameter can provide a significant advantage 
20 with respect to aspects of the present invention because, in many cases, a more recent 
earnings estimate or revision is likely to be more accurate than an estimate made 
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months prior to an earnings event. Estimates made prior to an earnings event may be 
classified according to Earliness Time Bins, where each Time Bin represents a range 
of days preceding an earnings event. For example, one Earliness Time Bin may 
include all estimates made between 0 and 90 days prior to an earnings event. 
5 By using this analysts* past performance database, the invention enables a 

user to rank, measure, and analyze analysts' historical performances based upon any 
metric, including a comparison of all or a subset of analysts within an Earliness Time 
Bin; a comparison of selected analysts across several Earliness Time Bins; 
scatterplots of percent errors versus number of days early for a single or plurality of 

10 analysts; and other comparisons. 

According to one embodiment, the invention allows for the rapid 
visualization and analysis of analysts' estimates or recommendations by creating and 
maintaining indices for predetermined data relationships, pre-calculating and storing 
predetermined analyst performance metrics, and calculating, compressing and storing 

1 5 time series estimates and summary measures of those estimates. 

According to another aspect of the invention, a front-end graphical user 
interface (GUI) is provided to facilitate analysis of analysts' prior performance for 
one or more securities. Preferably, the prior estimates are stored in a database that 
includes, but is not limited to, fields corresponding to a security identifier for each 
20 security; a plurality of earnings events, such as, for example, the issuance of a 
company's actual quarterly or yearly earnings reports; earnings event dates; an 
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analyst and broker identifier; and predetermined periods of times preceding an 
earnings event. Other fields and types of data also may be included. The front-end 
GUI will allow a user to select easily a security, earnings event, event date, Earliness 
Time Bin, and Contributors for analysis. The retrieved analysis information may be 
viewed as either raw data or, by using a data visualization technique, as a chart or 
graph. 

According to one aspect of this embodiment of the invention, each analysts 1 
estimates and revisions thereto are displayed simultaneously, along with the actual 
earnings of the companies they follow. Preferably, each analysts' estimate is plotted 
on a graph displaying the estimate (in dollars and cents) on the vertical or y axis and 
time (in days) on the horizontal or x axis. More specifically, each analyst's estimate 
is displayed as a horizontal line at a level corresponding to the estimate and having a 
length equal to the number of days that the analyst's estimate was at that level. If any 
analyst revises that estimate, a new horizontal line is displayed at the new level and 
the two lines are connected by a vertical line, such that the plot takes the form of a 
step function. 

Other levels of control may be provided including displays of derived time 
series such as high estimates, mean estimates, low estimates, and/or Composite 
estimates with or without a simultaneous display of actual earnings. To further 
facilitate the viewing of such data, each estimate, whether reflecting the 
determination of an individual analyst or a derived estimate, may be assigned a 
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unique color. A legend box may also be displayed simultaneously with a chart or 
graph that indicates which color is assigned to which estimate. Selecting an analyst 
or derived estimate series in the legend highlights (emboldens) the corresponding 
time series in the chart. The user may show or hide individual estimate series by 
5 means of on/off controls in the legend. The user may sort the legend by analyst 
name, broker name, or other criteria. Advanced navigation techniques include 
selecting an analyst from the legend and issuing a command (e.g., from a right-click 
pop-up menu) to jump to a detailed display of historical performance for the selected 
analyst. The user has the ability to arbitrarily change the scale of viewing and can 

10 zoom in to fill the screen with two days of information or zoom out to see five years 
of information. Optionally, a chart of the selected securities prices can be displayed 
on a chart below the estimates chart. The horizontal (time) axes of the two chart are 
synchronized so that zooming one chart zooms. This technique is valuable for 
understanding the impact on changes of estimates (or derived estimates) on changes 

15 in security price. Viewing historical estimates in this fashion may provide context 
and thus aid in the understanding of an analyst's performance track record and 
estimate revision patterns. This information can be used valuably when deciding 
how to appraise changes in an analyst's current estimates. This information can also 
be used valuably in building understanding of estimate and recommendation changes 
-20 in general and therefore help the user create more valuable autoweight models. 
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According to another embodiment of the invention, a user may rank, measure, 
and analyze the profitability of analysts' recommendations regarding the advisability 
of purchasing, selling, or holding a particular security at any given time. More 
specifically, the system allows a user to control, manipulate, and otherwise refine the 
normalization and translation of the recommendation descriptions of individual 
analysts to the scales published by FISPs such as First Call or IBES, which are used 
generally in the financial and business communities. In addition, weighting factors, 
similar to the ones described above relating to earnings estimates, and/or adjustment 
factors may be calculated for analysts' recommendations. Therefore, the system 
enables a user to view an analyst's "corrected" estimate through the use of the 
adjustment and weighting factor; The system also enables a user to compare and 
chart the profitability of following the recommendation of one analyst versus that of 
another analyst or the average recommendation. In addition, users -may create 
portfolio-creation rules to determine when and how much of a security to buy or sell 
and, furthermore, to track the value and test the profitability of having carried out 
such rules for a single or plurality of analysts over any given time period. 
Detailed Description of the Drawings 

FIG. 1 is a high-level block diagram of the security analyst performance and 
tracking analysis system in accordance with the present invention; 

FIG. 2 is a flow diagram illustrating the method for determining the 
adjustment factor for a particular analyst's estimate; 



13 



WO 99/56192 



PCT/US99/08909 



FIG. 3 is a flow diagram illustrating the method for determining the 
weighting factor for a particular analyst's estimate; 

FIG. 4 is a depiction of a graphical user interface used to input and designate 
analysis performance set parameters and metrics for displaying the historical earnings 
estimates and revisions of such estimates of a plurality of analysts and other related 
information; 

FIG. 5 is a depiction of a graphical user interface used to input and designate 
analysis performance set parameters and metrics for displaying either the raw data or 
visual display of a number of representations of the historical accuracy of the 
earnings estimates of either a single or a plurality of analysts; 

FIG. 6 is a sample illustration of the graph produced by actuating the GUI of 

FIG. 4; 

FIG. 7 is a sample illustration of a graph produced by actuating the GUI of 

FIG. 5; 

FIG. 8 is a sample illustration of a graph produced by actuating the GUI of 
FIG. 5; and 

FIG. 9 is a sample illustration of a graph produced by actuating the GUI of 

FIG. 5. 

FIGs. lOa-c depict system architectures according to various embodiments of 
the present invention. 
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FIGs 1 la-c depict graphs illustrating comparative performance by two 
analysts with actual outcome. 

Fig. 12 depicts a screen display of a analyst search screen according to an 
embodiment of the present invention. 

Fig. 13 depicts a screen display of a broker search screen according to an 
embodiment of the present invention. 

Fig. 14 depicts a screen display of a stock filter and list screen according to an 
embodiment of the present invention. 

Fig. 15 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 

Fig. 1 6 depicts a screen display of a model manager according to an 
embodiment of the present invention. 

Fig. 1 7 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 

Fig. 18 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 

Fig. 19 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 

Fig. 20 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 
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Fig. 21 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 

Fig. 22 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 
5 Fig. 23 depicts a screen display of a model manager screen according to an 

embodiment of the present invention. 

Fig. 24 depicts a screen display of a model manager screen according to an 
embodiment of the present invention. 

Fig. 25 depicts a screen display of a backtester screen according to an 
1 0 embodiment of the present invention. 

Fig. 26 depicts a screen of a historical view of analyst's estimates over time 
according to an embodiment of the present invention. 

Fig. 27 depicts a screen of a historical view of analyst's estimates over time 
according to an embodiment of the present invention. 
15 Fig. 28 depicts a screen display of an analysis screen according to an 

embodiment of the present invention. 

Fig. 29 depicts a screen display of various source selector screens according 
to an embodiment of the present invention. 

Fig. 30 depicts a screen display of a performance screen according to an 
.20 embodiment of the present invention. 
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Fig. 3 1 depicts a screen display of a performance screen according to an 
embodiment of the present invention. 

Detailed Description of the Preferred Embodiments 

FIG. 1 is a block diagram illustrating a security analyst performance tracking 
and analysis system according to one preferred embodiment. In addition, FIG. 1 
shows, in a broad sense, the data flow occurring within said system during a typical 
query for an analyst estimate comparison. 

On an interim basis, which can occur either daily, monthly, or at any other 
period, a Global Analyst Data Object, containing historical data on analyst estimates, 
may be transferred or otherwise downloaded, through a telecommunications link or 
similar method or device, from an Originating Storing Object, such as a commercial 
database maintained by any one of a number of financial information service 
providers (FISP) such as IBES, Inc. (IBES) or Fist Call Corporation (First Call), to a 
Retrieving Storing Object, which will preferably, but not necessarily, take the form 
of a network server in a computer network such as, for example, a local area 
network. 

The Global Analyst Data Object downloaded from the Originating Storing 
Object is typically comprised of numerous files and fields relating to historical data 
relevant to analyst earnings estimates and recommendations, An example of such a 
historical database is that maintained by financial information services provider 
IBES. IBES's database, known as the Daily Detail Earnings Estimate History, 



17 



WO 99/56192 PCTAJS99/08909 



contains more than ten years of analyst estimate forecasts, compiled from data 
obtained from more than 200 brokerage houses and more than 2000 individual 
analysts, for United States companies. These files and fields contain both general 
and specific information on analyst estimates and related data including, but not 
5 limited to, information pertaining to financial instrument type and related 
identification codes, broker and analyst identification, industry groupings, and 
detailed information on such variables as the prices of particular securities on 
specific dates. Importantly, it should be noted that a Global Analyst Data Object may 
be used which contains analyst data pertaining not only to stocks publicly traded in 

10 the United States, but also international stocks and any other type of financial 
instrument currently in existence or created in the future. 

Either during or after the downloading of the Global Analyst Data Object, the 
database files are manipulated and otherwise processed such that they are 
restructured according to predetermined data fields, thereby creating a Restructured 

15 Analyst Data Object. In this way, each analyst earnings estimate and 
recommendation is restructured to have a number of predetermined data fields. 

As indicated above, the data comprising the Restructured Analyst Data 
Object will preferably reside on a server in a computer network. Using a computer 
terminal or other similar input device, a user will be able to access and utilize the 

20 application Module comprising the software for the present invention. This Module 
may or may not reside on the computer terminal operated by the user. 
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In a preferred embodiment, the architecture of the present invention may 
comprise various structures consistent with the present invention. Various structures, 
as depicted in Figs. lOa-c may be provided. These systems may comprise tiers such 
as in an Internet based networking environment. The tiers may comprise a 
Presentation Layer, depicted in FIGs. lOa-c as a plurality of terminals. This tier may 
be operatively connected to a second tier known as the Application Layer, depicted 
as the Retrieving Object. Additionally, the Application Layer is preferably 
operatively connected to a third tier, or Data Layer, which is depicted as the 
Restructured Analyst Data Object. Other tiers may also be provided as depicted in 
the Figures. 

* In a preferred embodiment, historical data accessed from the Originating 
Storing Object is stored on the Restructured Analyst Data Object in addition to pre- 
calculated analyst-performance metrics derived from the Originating Storing Object. 
Because the data acquired directly from the Originating Storing Object may be 
proprietary of the data provider, the Retrieving Object preferably selectively delivers 
proprietary data only to those terminals having a license for the proprietary 
Originating Storing Object data. Also, the Retreiving Object preferably delivers 
only the pre-calculated analyst performance metrics derived from the Originating 
Storing Object to those terminals not having a license for the proprietary data. Using 
such a terminal and application module, a user may access and query the 
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Restructured Analyst Data Object to perform any one of the numerous functions able 
to be performed by the present invention. 

The pre-calculated data may comprise error metrics for each security, for each 
historical fiscal period in the database, for each contributor (e.g., analyst/broker pair) 
5 is a row in the database. Each row comprises multiple error metrics valued over a 
range of time periods. Error metrics may comprise various metrics including a raw 
error indicator (analyst estimate minus the actual earnings for a particular event), 
error percent to actual earnings, percent available (percent of time that an analyst had 
an estimate or recommendation available in the relevant time frame), error percent to 
10 consensus, and the number of days between the estimate of an event and the actual 
event, for example. Average error metrics may also be stored. Table 1 below 
provides one embodiment of average error metrics that may be maintained as well as 
other metrics that may be stored. 
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The calculations to derive these error metrics are provided in Table 2. 
Example ranges, analysis of these values and characteristics are provided although 
other ranges, analysis and characteristics may also be provided. 
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Additionally, other metrics including leadlag factor, swings, hits, hit percent, 
and mean time between revisions may be included as metrics. Table 3 below 
described these metrics, how they are calculated, analysis for these metrics, and a 
range for these metrics. 



24 



WO 99/56192 



PCT/US99/08909 



o 
e 



CO <U ° 

c* £ O 



~ oo 

CO t) 
O £ 



o 
o 



CO 

to 



GO 



CO 
CO 



0) ^ 
CO v 

CO 

2 & 
o < eo 

u J. 5- 



CO S 
CO O 
<L> f H 

Q. w 

x c 
5 c ^-^ 



c 

1 M 

CO 



I £1 

< s r 

CO CO 

*-> 00 >> 

^ > c 

CO CO rt 

cr o 



to CO 



CJQ 



eg o w 

.si" 



O 



1 e 



5 * 



CO 



co ~ 



s a a 



eo 
o 

— -5 

n© C 

6 s - • 

O v o 
O 

— . O 



CO 

>> 

CO 



On 
00 
CO 



CO to 

CO g 
<D co 



3 



CO 
CO 



c 

CD 
T3 



2 

1 

CO 

s: 

I 



ao 
C 

CO 

< 



.2 
co 
cr 



3 
3 



3 



II 

z 

c 

cO 



> 

a 

-a 



Q co £ 
^ ^ 00 

3 ~ 



Q co 



CO 
-C 

CO 

a 

CO 

CO 

.22 

< 



c 
I 



3 

CO 

C 

<D 
CO 

C 

o 
o 

V 




o 
o 

X 



I 

s 



3 



8 

00 

s 

•3 

CO 



o 

•a S 

.2 . 

o- o 
.5 

o 

CO d 

3 O 
^ 3 
O to 

»L 

X) co 

6 -S 

co 3 52 

00 c c 
c „ « 

5 « « 
co *o -5 



CO 



> 

Q 



I 

to 

c 

o 

co -2 
.5 "o 



a) o 
3 H 

CD CO 

E 

g.§ 

g « 

CO u. 

C 4> 

O <C 

O CO 



o 

Cm 



CO 

C 



25 



WO 99/56192 



PCT/US99/08909 



These metrics are understood as follows: 
Error $ - The difference between Est ( and the Actual. Expressed in dollars. 
Abs Err $ - The absolute value of Error $ at a point in time. 

Bias Error Percentage - If Consensus > Actual, then Bias Error equals Relative Error 
5 %, else it is 0. If Consensus < Actual, then Bias Error equals Relative Error %, else 
it is 0. 

Actual-Divisor (Applies to Err%, ABS(Err%), and RelErr%) - To facilitate cross- 
stock and cross-period comparison of error, we provide metrics that normalize 
estimates & error by the size of the actual earnings. Of course, for small actual 
10 values, errors become exaggerated. To avoid this, we limit the divisor to be no less 
than .40 cents for fiscal year events and no less than . 1 0 for fiscal quarter events. 
Relative Error Percentage - The difference between the analysts error and the 
consensus error, divided by the Actual-Divisor. 

Swings - Often, major revisions ( N Std Dev away from consensus) occur 
1 5 simultaneously for multiple analysts. For example, this may be the case when a 

company reports a large earning surprise or issues a warning about upcoming growth. 
"Swings," which are bold estimates that differ greatly from the consensus, are 
differentiated from major revisions that occur concurrently with, or near to, major 
revisions from other analysts. To achieve this, the system may measure whether an 
20 analyst estimate or revision is N standard deviations away from the consensus N 
(typically 5) days after the day the analyses estimate was made. Swings may be 
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measured over the 24 months prior to the report date. Unlike other error metrics 
which are calculated by sampling (continued) estimates over an interval and 
computing the corresponding average error, Swings may be determined by 
considering only the actual estimates or revisions. The default number of Std Dev is 
5 1.5. 

Hit Percent - A hit is a swing that proves to be closer to the actual than the consensus 
at N days after the date of the swing. 

Total Estimates - The total number of estimates made by the analyst in the prior 24 
months for the event. Confirmations are not included. An estimate pre-existing 
1 0 exactly 24 months prior to the report are counted in the total. 

Follow Percent - In each time frame (0 to 3, 3 to 6, 6 to 12, 0 to 12, 0 to 24 months) 
we calculate the total availability of the analysts estimates during that time. Follow 
Pet equals the days the analyst estimate was available in the timeframe divided by the 
total number of days in the timeframe. 
15 MTBR - Mean Time between Revisions - Measures frequency of analyst revision in 
the year prior to the report date. Equals the number of days in which there was an 
active estimate in the year prior to the report date divided by the Total Estimates. 
Best Date - The day in which the analyst's error (RelErr%) was lowest in the 24 
month prior to the report date for that event. 
20 Best Error - The value of the analyst's lowest RelErr% at the corresponding Best 
Date. 
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Further, a lead lag score may be provided. In calculating the lead lag score, 
Table 4 represents calculations with the following understanding; 
Co represents the consensus on the day of the estimate in question, Ci represents the 
consensus on the n-th day prior to the day of the estimate in question, and C2 
represents the consensus on the n-th day following the day of the estimate in 
question. These conditions are considered in this order to determine if an estimate is 
leading, lagging, or neither: 
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For each analyst, each new estimate or revision made within 24 months of a 
report date for a fiscal period is classified either as Leading, Lagging or Neither 
according to the logic above. The LeadLagFactor is the number of Leading estimates 
minus the number of Lagging over the total estimates. If all estimates were lagging, 
5 the LeadLagFactor^ -1.0; if All estimates were leading, the LeadLagFactor = +1 .0. If 
all estimates were "neither" or if the number of Leading Estimates equals the number 
of Lagging estimates, the LeadLagFactor = 0.0. Estimates already current at 24 
months prior to the report date may not be included. 



Leads — Lags 

10 LeadLagScore = 



TotalEstimatesForA nalyst 
Based on the information in this database, various other calculations may be 
derived. For example, based on the historical information for each analyst, an 
adjustment factor may be calculated. Generally speaking, the adjustment factor 
represents the analytical "bias" which may or may not be incorporated into each 
15 analyst's earnings estimate, for a particular security, over a given period of time. For 
example, an analyst who has, over a specified time period, issued earnings estimates 
for a particular company that were, in hindsight, on average too high, might be 
assigned an adjustment factor of 0.95 for that performance analysis set, such that the 
analyses issued estimate over the specified time period is reduced by five percent. 
20 Conversely, an analyst who has historically issued estimates over a specified time 
period that were, in hindsight, on average too low might be assigned an adjustment 
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factor of 1.10 for that performance analysis set, such that his actual reported estimate 
for that time period is increased by ten percent. 

Notably, although the adjustment factor calculated for any given performance 
analysis set may be stored in the system's database, adjustment factors are typically 
generated in real time in response to user-defined inputs. As indicated above, the 
calculation of an adjustment factor will generally be based upon a comparison of the 
historical earnings estimates issued by an analyst, for a given security followed by 
that analyst, over a particular time period. A user may also define further analysis 
parameters and metrics such that, for example, as specified by a user, the 
determination of an adjustment factor may take into account an analyst's historical 
percentage error as compared to actual earnings, generally available consensus 
earnings estimates, custom composite adjusted earnings estimates, or other metric. 

One example of a user-defined parameter is the assignment by a user of a 
scaling factor to be applied in the calculation of the adjustment factor for a given 
performance analysis set. For example, a user may define a performance analysis set 
such that, for that analysis set, a particular analyst is shown to have issued estimates 
that were on average 20 percent greater than actual earnings. The user will then be 
able to assign a scaling factor, say for example, 0.5, to be multiplied by the 20 
percent error such that the effective adjustment factor for that user-defined 
performance analysis set reflects a 10 percent and not a 20 percent adjustment — i.e., 
an adjustment factor of 0.9, rounded to the nearest tenth. Thus, in this particular 



31 



WO 99/56192 



PCT/US99/08909 



example, the user "discounted" the analyst's earnings estimate bias as indicated by 
the system's calculations. The formula for the calculation of the adjustment factor is 
set forth below: 

[ 1 / (1 + (Error metric * Scaling factor ))] 
5 An improved estimate of an analyst's earnings may also be accomplished by 

the calculation of a weighting factor which is used to provide a weighted average of 
an analyst's earnings estimate, as compared to other analysts. As with the adjustment 
factor, this weighting factor may be based upon a variety of user-defined parameters 
and metrics. For example, a weighting factor for a given analyst, security, and time 

10 period may be calculated based upon the relative recency of the issuance of analyst's 
earnings revision or the historical consistency and/or accuracy of an analyst's 
adjusted estimates (as compared to actual earnings), or a combination of these or 
other related factors or metrics. For example, if an estimate of one analyst is 
relatively old compared to an estimate or revision of a second analyst, the former 

15 might be assigned a relatively low weighting factor, (or even zero in some cases), as 
compared to a more recent estimate produced by the second analyst. This is done 
based upon the assumption that a more recent estimate is likely to be based upon 
relatively new and accurate information which may affect a company's earnings 
potential and, therefore, is more likely to be predictive of a company's actual 
20 earnings. 
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Similarly, a weighting factor for a given analyst, security, and time period 
may be calculated based upon the relative accuracy of one analyst as compared to 
another. For example, at a specific point in time prior to an earnings event for a 
specific security, analyst A might have issued an earnings estimate 25 percent greater 
than the actual earnings ultimately announced, whereas analyst B may have issued an 
estimate that was 100 percent greater than the actual earnings. Subsequently, at a 
later point in time, yet still prior to the announcement of actual earnings, analyst A 
might have revised his estimate so that it was 15 percent greater than the actual 
earnings, and analyst B may have simultaneously revised his estimate so that it was 
80 percent below actual earnings. Although the average errors for both analysts A 
and B were 20 percent above the actual earnings, the variance over time for analyst A 
was much less than that for analyst B. Accordingly, analyst A's estimate of future 
earnings for this specific security might be assigned a weighting factor significantly 
higher than that assigned to the estimate of analyst B. 

Regardless of how a weighting factor is calculated, based on the number of 
analysts being tracked, the total value of all weights will equal one. As a result, the 
weighting factor assigned to an analyst for a given performance analysis set is really 
a distribution number that is evaluated in the context of a set of a plurality of analysts 
issuing estimates regarding a particular earnings event for a particular security. 
Importantly, as indicated above, the weighting scheme employed can be controlled 
and altered by the user. 
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The adjustment and weighting factors described above may be used, together 
with an analyst's actual earnings estimate, to calculate a custom composite estimate 
to arrive at a more accurate estimation of a company's earnings. A custom 
composite estimate is calculated by multiplying an analyst's current earnings 
5 estimate (for a given security, time period, and other parameters) by its 
corresponding adjustment and weighting factors for that given performance analysis 
set. The results for each estimate for each analyst being studied are then summed to 
arrive at the custom composite estimate. It will be appreciated that the calculation of 
a custom composite estimate provides investment managers and similar users with a 

10 way of better predicting not only the accuracy of an analyst's earnings estimates but 
also the actual earnings of a company over any given period of time. 

As indicated above, a predetermined system database is constructed such that 
each analyst estimate record in the database contains unique fields related to that 
estimate. In general, these records contain a combination of data fields present 

15 within the Global Analyst Data Object obtained from the FISP and data fields unique 
to and created within the system of the present invention. Typically, the fields in this 
restructured database may include an analyst identifier; an event identifier 
corresponding to a specific security; an event type and date (e.g., Apple, FY- 1995 or 
Intel, Q2-1997); an estimate date; a raw error indicator which corresponds to an 
20 analyst's estimate minus the actual earnings for a particular event; other metrics such 
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as the percent error from an analyses estimate to either the actual earnings or the 
consensus error; or other error metrics defined by a user. 

The typical system database record will also contain an Earliness field which 
contains the number of days by which an analyst's earnings estimate precedes a 
particular earnings event, such as a company's quarterly or annual earnings postings. 
This Earliness field can be employed to group estimates of similar Earliness into like 
Earliness Time Bins. It will be appreciated that this Earliness field or the usage of 
Earliness Time Bins will likely enhance numerous aspects of the present invention 
because, in many circumstances, the accuracy of an estimate made shortly before an 
earnings, event is likely to be more accurate than an earnings estimate made months 
prior to the earnings event. Specifically, each Earliness Time Bin represents a range 
of days early such that prior analyst estimates may be classified according to 
particular Earliness Time Bins. For example, one Earliness Time Bin may include 
all estimates issued by a specified group of analysts for a given security between 7 
and 30 days, inclusive, prior to an earnings event or between 31 and 90 days before 
such an event. In this way, this unique field will allow users to make meaningful and 
valuable comparisons between analyst estimates for any number of given time 
periods preceding a particular earnings event. 

Importantly, in addition to the predetermined data fields discussed above, the 
database of the present invention may also contain and maintain indices for 
predetermined data relationships and predetermined analyst performance metrics for 
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a plurality of analysts, such as time series estimates and summary measures of those 
estimates. Accordingly, by utilizing this restructured database, a user will be able to 
both rank and analyze the performance of a plurality of analysts based upon any 
metric. Moreover, based on the data contained in the system database, the present 
5 invention allows for the rapid visualization of the analyses of analysts* earnings 
estimates and buy-sell recommendations. 

A front-end graphical user interface (GUI) is provided, examples of which are 
shown in FIGs. 4 and 5. Referring to FIG. 4, the graphical user interface shown 
contains a plurality of graphical buttons and selection boxes with pull-down menu 

10 capability. Preferably, the selection boxes correspond to predetermined fields 
contained within the system database such as a security identifier or ticker; an 
analyst; a plurality of event types, such as yearly and quarterly earning postings; 
earnings event dates; and dates corresponding to Earliness Time Bins. Notably, 
however, other fields and types of data may be included. Also, the GUI and system 

15 allow a user to manually input specific data not initially present in the database for 
analysis purposes. Manual inputs made by a user are thereafter stored within the 
system database such that the system may recall and list a user's prior inputs as part 
of a selection box's pull-down menu of selection alternatives. In this way, the front- 
end GUI allows a user to easily select from a large number and wide variety of 

20 analysis parameters and metrics. 
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According to one aspect of this embodiment, the historical earnings estimates 
of a single or plurality of analysts may be graphed using the GUI shown in FIG. 4. In 
this embodiment, a GUI similar to the one discussed above is used except that, in 
addition to the selection boxes and graphical buttons described above, there are 
graphical buttons present which allow a user to view a summary of the actual, high, 
low, and average consensus estimates as derived from the earnings estimates 
provided by all analysts within the system database. It is anticipated that at least one 
analyst can be excluded from a calculation using an exclusion function, described 
further below. 

Actuating the GUI display of FIG. 4 may generate, for example, the graph 
shown in FIG. 6. Referring to FIG. 6, it can be seen that the earnings estimates, and 
revisions of those estimates, of each analyst chosen in FIG. 4 are displayed 
simultaneously on a graph where the horizontal axis corresponds to a user- 
determined period of time or Earliness Time Bin, (preferably showing time in days) 
and the vertical axis plots the range of earnings estimates, (preferably in dollars and 

r 

cents). More specifically, each analyst's earnings estimates and revisions over time 
are displayed as horizontal lines corresponding to and level with that analyst's 
earnings estimate for a particular time period. When an analyst makes a revision to 
an estimate, whether upward or downward, the change is plotted as a step function 
with a vertical or essentially vertical line connecting the two horizontal lines 
representing the difference between an analyst's earlier and revised estimates. The 
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length of each horizontal line equals the number of days, as displayed on the 
horizontal axis, that the analyst's estimate was at that level. As dictated by a user's 
selection of analyses inputs from the GUI of FIG. 4, such a graph may also display 
the high, low, and average consensus estimates along with the estimates of the 
5 analysts specified by the user. Additionally, one or more vertical bars showing the 
actual earnings for the relevant security at specific earnings posting dates may also be 
displayed. To facilitate reading and interpreting the graph, each analysts' earnings 
estimate, as well as the high, low, and consensus estimate, and actual earnings bars 
may be displayed in a unique color. To further facilitate reading and interpreting the 

10 graph, a legend box may be displayed simultaneously with said graph which shows 
the colors associated with each estimate displayed. 

It will be appreciated that viewing the historical estimates of a plurality of 
analysts in the manner described above may often provide a context within which an 
individual analyst's estimates and revisions can be better understood, such as by 

15 providing insight into an analyst's estimate revision patterns arid the relative 
accuracy of those revisions over time as they relate to a company's actual earnings 
postings. As such, this information will likely be valuable in appraising specific 
revisions made by an analyst to his current estimates, and in deciding whether to act, 
or to not act, based upon the revisions. 

20 According to another embodiment of the invention, the accuracy of analysts' 

estimates over a single or plurality of time periods, for any given earnings event, can 



38 



WO 99/56192 



PCT/US99/08909 



be ranked and visually displayed. Specifically, referring to FIG. 5, a GUI is provided 
similar to the one shown in FIG. 4. In addition to containing selection boxes and 
graphical buttons pertaining to a security identifier or ticker, analysts, a plurality of 
event types, event dates, and dates corresponding to Earliness Time Bins, also 
included are selection boxes corresponding to the number of estimates made by an 
analyst; specific analysis metrics, such as raw error or percent error as compared to 
actual earnings; and average and standard deviation metrics. 

The information retrieved using the GUI in FIG. 5 may be viewed either as 
raw, numeric data or, by using a data visualization technique, as a chart, graph, or 
combination thereof. As shown in FIG. 5, a user can specify a visualization 
preference by choosing from and actuating a particular GUI button, such as "view 
chart" or "view data." For example, FIG. 7 shows a bar chart, created by a user 
actuating the "view chart" graphical button in FIG. 5, illustrating a comparison of the 
average percent error of a plurality of analyst estimates, made in the 6 to 10 months 
prior to the end of Apple Computer's fiscal year. Preferably, all those analysts 
chosen by a user from the GUI of FIG. 5 are displayed simultaneously along the 
horizontal or y-axis. As shown, the vertical or x-axis displays a measure of the 
average percent error, both positive and negative, of the estimate of each analyst 
displayed as compared to the actual earnings for a given security. This visual display 
allows a user to essentially rank individual analysts by the accuracy of their estimates 
for a given period of time or Earliness Time Bin, prior to an earnings event, and to 
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identify the analyst(s) with the most accurate earning estimate for a given security, 
earnings event, and preceding time period. In addition, this visual display clearly 
illustrates the earnings bias of individual analysts such that, patterns, if any, in an 
analyst's earnings estimations may be investigated and analyzed. 
5 Additional information could also be incorporated into such a graphical 

display. For example, the vertical axis may display a measure of the average percent 
error, both positive and negative, of the estimate for each analyst displayed as 
compared to actual earnings. However, the bar representing and analysts estimate 
also shows, in addition to the percent error of the analyses issued estimate, the high 

10 and low estimates, or percentile errors estimates (e.g., 90 th and 10 th percentile errors 
estimates), published by that analyst over the performance analysis set. A black 
section of an analyst's graphical bar may represent the average error for that analyst. 
Extending vertically above the black section is a bar segment which ends at a level 
representing that analyst's high estimate over the performance analysis set. 

15 Similarly, extending vertically below the black section of the analyst's bar is a bar 
segment which ends at a level representing that analyst's low estimate over the 
performance analysis set. Additionally, an analyst legend box may be displayed for 
each analyst which may show such information as the number of years an analyst has 
been providing estimates for the security in question, and the first and last period in 
.20 which the analyst issued such estimates. In this way, the individual analyst legends 
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may provide further historical context to the historical error performance graph of 
FIG. 7. 

These graphs, such as the one in Fig. 7, may be generated by a user 
designating the performance set for (a) each analyst in list and (b) for each event 
chosen. Next, the system user fills estimate array with data, for all chosen analysts, 
to create estimate time series data array — e.g., estimate. After that, the system fills 
error array, for all chosen corresponding events and dates, to create time series error 
array — e.g., error. Next the system loops for each event chosen in the performance 
analysis set back through the previous steps. The system then calculates an error 
summary metric per analyst — e.g., average error - such as average error, standard 
deviation, 10 and 90 percent high and low etc. and does so for each analyst. The 
chart may thus be provided. 

Similarly, a visual display may also be generated which illustrates a 
comparison of analysts' performances over not just one but rather a plurality of time 
periods, as shown in FIG. 8. Here, the Earliness Time Bins chosen by the user are 
displayed in chronological order, with the most recent time period beginning on the 
left. The vertical axis displays a measure of the average percent error, both positive 
and negative, of the estimate of each analyst chosen by a user for analysis. A bar 
chart is generated wherein each analyst's percent error is indicated by a different 
color bar. In this way, a user can track a plurality of analysts' earnings estimates over 
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time so as to determine when individual analyst's estimates are more accurate as 
compared to others. 

The graph shown in FIG. 9 may also be generated by using the GUI of FIG. 5. 
Unlike the graphs of FIGs. 7 and 8, which plot the average percent error of analysts' 
5 earnings estimates as compared to other analysts for given periods of time, the graph 
of FIG. 9 displays a scatterplot of the error of a single analyst's estimates relative to 
the consensus estimate error at the time the analyst's estimates were issued. Here, 
the horizontal axis shows the number of days prior to the user-defined event, whereas 
the vertical axis shows the error relative to the consensus error. A value of one on 
10 the vertical axis corresponds to an estimate error that is equal to the consensus error. 
Each scatterpoint represents an estimate or revision made by an individual analyst at 
a specific date. Although not shown in FIG. 9, vertical lines perpendicular to the 
horizontal axis may be superimposed on the scatterplot chart to indicate the dates of 
various earnings events. It will be appreciated that by utilizing such a scatter display, 
15 a user may be able to ascertain at a glance which analysts are more likely, either in 
general or at specific points in time, to publish estimates that are more accurate than 
the current consensus estimate. Importantly, it should be noted that the graph of FIG. 
9 may be used to display a scatter chart of any metric including, but not limited to, 
raw error, the percent error as compared to actual earnings, or other user-defined 
120 errors. 
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In another embodiment of the invention, a user will be able to construct, 
store, and recall custom composite earnings models for analysis and testing purposes. 
Specifically, the system will allow for users to create financial transaction models by 
inputting specific performance analysis sets, including performance metrics and other 
user-defined metrics and parameters, such as scaling factors, to arrive at specific 
custom composite earnings estimates. These performance analysis sets and 
corresponding custom composite estimates may then be stored in the system's 
database for later retrieval. In this way, a user will be able to test such models by 
applying them over any previous time period, thereby essentially creating a "virtual 
analyst" whose hypothetical prospective performance can be compared with the 
historical performance of a single or plurality of analysts, or even the average 
historical consensus estimates for any previous time period. Most significantly, it 
will be appreciated that by conducting such tests a user may be able to refine a model 
that can be used to accurately predict the accuracy of prospective, analysts' earnings 
estimates. 

FIGS. 1 la to 1 lc depict the purpose between the two different types of error 
calculation. In FIG. 1 la, two analysts have made predictions concerning the earnings 
of a particular security. Their predictions, in dollars, are shown on the y-axis where 
So is the actual earnings, whereas the time at which the analysts made their 
predictions is shown along the x-axis. The difference between each of the depicted 
adjacent markings on the y-axis is equal to $, and the difference between each of the 
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depicted x-axis markings is equal to T. The first analyst initially (t 0 ) predicted above 
the actual earnings by $2, and at time ti modified the prediction to an estimate below 
(-$2) the actual earning. The second analyst predicted earnings slightly below the 
actual earnings for the entire period shown. 
5 Turning to FIG. lib, which highlights the error associated with the first 

analyst's predictions, it is shown that the first analyst has an average error equal to 
zero because the extent of the overestimate is approximately equal to the extent of 
the underestimate. This raw error metric is preferably calculated as follows: 

tn 

^ (Estimate - Actual) 
to 

10 By substituting the values shown in FIG 1 lb, the overestimate is found to be 

($2 - $o)*(ti - to) or 2$T and the underestimate is found to be (-$2 - $o)*(t2-tt) or - 
2$T. Accordingly, the first analyst would receive a raw error of 0 and would 
accordingly be given no adjustment factor. In determining the weighting factor, 
however, the following equation which represents the absolute error metric, is 

15 preferably used: 

tn 

^{(Estimate - Actual)] 
to 

Again substituting the values for the first analyst, an absolute error of 4$T is 
found. Applying the same analysis to the second analyst leads to a raw error of -2$T 
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which could in turn be used to calculate an adjustment factor. Similarly, because the 
second analyst consistently underestimated the actual earnings, the second analyst 
would have an absolute error of -2$T. Because the absolute error of the second 
analyst is half as great as the absolute error of the first analyst, the second analyst is 
5 preferably assigned a weighting factor greater than the weighting factor of the first 
analyst. 

Because analysts start making predictions on a given security at different 
times, it is possible that a particular analyst will not have made predictions about a 
particular security for the entire duration over which an error analysis is being 

10 performed. In a preferred embodiment, it is possible to make proportional 
adjustments to various error analysis based on the percentage of time that a given 
analyst has been tracking a security. 

Similarly, because analysts start making predictions on earnings at different 
times, it is similarly possible that certain analysts will not have made earnings 

15 estimates at a time when an unanticipated event lead to a significant error. In a 
preferred embodiment, the effect of such unanticipated events can be filtered by 
comparing the analysts predictions to a consensus estimate. Such a comparison is 
termed a relative error metric. The following equation provides an example of a 
relative error metric: 
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^ (| Estimate- Actual\-\Consensus - Actual\) 
~ Actual 

The relative error metric shows how a particular analyst performed in relation to the 
other analysts who were tracking a particular security over the analyzed period of 
time. The purpose of utilizing the actual earnings in the denominator of a preferred 
5 embodiment is to enable errors to be normalized so that comparisons can be made 
across different securities. Because small actual earnings can lead to exaggerated 
errors, it is possible to establish a minimum actual value, for purposes of this error 
metric, to prevent such exaggerated errors. For example, if the actual earnings were 
0, then any analyst tracking the security would have an infinite error, so a value of, 

1 0 for example, $.40 could be used to provide useful information from the analysis. 

In another embodiment of the invention, a user will be able to rank, measure, 
and analyze the historical accuracy of a single or plurality of analysts' buy-sell 
recommendations in various ways. As an initial matter, a user will be able to control 
and otherwise define how recommendation descriptions used by a plurality of 

15 analysts are normalized and otherwise translated into scaled recommendation 
numbers. 

Specifically, depending on the employer of an individual analyst, said analyst, 
when either upgrading or downgrading a particular security, will use varying 
descriptions to make his recommendation. For example, analysts at the investment 
20 firm Alex Brown issue recommendations using the following descriptions, 
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predetermined by the firm: strong buy, buy, neutral, source of funds, or sell. In 
contrast, analysts at the investment firm Goldman Sachs issue recommendations 
using the following descriptions, also predetermined by the firm: priority list, 
recommended list, trading buy, market outperform, market perform, and market 
under-perform. FISPs such as First Call translate and otherwise normalize the 
recommendation descriptions of the numerous analysts to a scale ranging from 1 to 5, 
with the following descriptions: 1 (buy), 2 (buy/hold), 3 (hold), .4 (hold/sell), and 5 
(sell). The FISPs then calculate an average recommendation by calculating the mean 
of all analysts' current recommendations as translated to this 1 to 5 scale. 

In the present invention, recommendation adjustment and weighting factors 
may be calculated in a way closely resembling that described above for analyst 
earnings estimates. For example, relatively recent recommendation upgrades or 
downgrades may be assigned a relatively high weighting factor while older 
recommendations may receive a weight of zero. Similarly, using these factors an 
improved custom composite recommendation may be determined which more 
accurately reflects the action (e.g., buy, sell, hold etc.) that a user should take with 
respect to a security. In addition, a user will have the ability to control the 
recommendation normalization process, if so desired, to replace the normalization 
performed by an FISP. 

Moreover, using either the FISP generated recommendation scale or user 
defined scale, a user will have the ability to measure the historical profitability of a 
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single or plurality of analysts 1 recommendations in much the same way as described 
above for analyst estimates. For example, using a GUI similar to FIGs. 4 and 5, a 
user can create a graph illustrating the average percent error of an analyst's 
recommendation as compared to the average recommendation. 



Specifically, a user can choose a security and then set up purchase and/or selling 
instructions that the system will make automatically. For example, a user can 
instruct the system to purchase a security when ' a specific analyst issues a 
recommendation of "2," double his investment if the recommendation is upgraded to 
10 "1," and sell all or a certain percentage of the security if and when the analyst 
downgrades his recommendation to "3" or lower. 

FIG. 9 provides an example of a scatterpiot graph created with the present 
invention. This scatterpiot is generated using the following equation: 



15 where bias error is equal to relative error if relative error is greater than the consensus 
error. If the relative error is less than the consensus error, then the bias error is 
assigned a value of zero over the selected time period. The consensus error is 
calculated the same as raw error is calculated for an individual analyst, except that 
the consensus estimate is used instead of the analyst's estimate. The bias error is 



Users will also have the ability to create and test portfolio creation rules. 
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useful in determining how consistently a given analyst or group of analysts 
outperforms the consensus for a particular security. 

Another option available in a preferred embodiment is the ability to exclude 
at least one Analyst. For example, if a particular analyst had an extreme error during 

5 a period of analysis that a user is evaluating* then the consensus error might be too 
reflective of that individual analyst's error. Accordingly, a majority of analysts could 
have bias errors approximately equal to zero which indicates that they are 
outperforming the consensus estimate. If a user wants to filter out an analyst's 
estimate for this or any other reason, it is possible to exclude the analyst's estimate 

0 from a particular metric analysis. 

In a preferred embodiment, there are additional metrics which can be used to 
evaluate how effectively an analyst acquires and reacts to information. One metric 
that serves to accomplish this task is the leadlag Factor. Preferably, the leadlag 
Factor is calculated as follows: 

{Leads- Lags) 

) TotalEstimates 

where leads is the number of times that an analyst makes an estimate revision before 
the majority of the analysts following a particular security, lags is the number of 
times that an analyst makes an estimate revision after the majority of the analysts 
following a particular security, and total estimates represents the number of 
predictions that the analyst has made. In a preferred embodiment, a user can select a 
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leadlag factor based on a number of different variables, including which securities, 
which analysts, which time periods, which earliness bins, or any combination thereof. 

Another metric that is useful in predicting how an analyst acquires and reacts 
to information is the hit percent. A hit percent is an evaluation of the number of 
5 times that an analyst successfully revises earnings. In a preferred embodiment, a 
swing is preferably an estimate that is outside a predetermined standard deviation of 
the mean of the consensus estimate. In a most preferred embodiment, a 
predetermined standard deviation of the consensus estimate is approximately 1.5. A 
hit is preferably a swing in which the analyst's estimate is closer to the actual 
10 earnings than the consensus estimate. A hit percent can then be determined by 
dividing the number of hits by the number of swings, and multiplying the result by 
100%. 

As discussed above, the system may provide the user with the option of 
viewing a large amount of information in a variety of different formats. 

15 According to one embodiment, the user may desire to view contributors to 

see relationships between stocks, analysts and brokers, as for example, shown in 
Figs. 12-14. In Fig. 12, the user may view by analyst, whereby the system presents 
analysts in the system and enables drilling into each analyst to see which firms they 
are associated with and which tickers they have followed. In Fig. 13, the user may 

20 search for a broker, and the system presents the brokers and enables drilling into the 
broker to view the analysts that work at the firm and the tickers followed by the 



50 



WO 99/56192 



PCT/US99/08909 



analysts and the broker. In another screen, the user may search for a ticker such as a 
stock. 

The system may also provide screens through which the user may manage 
stocks and stock groups against which analysis may be run. The user may do so 
through stock lists and stock filters. Stock filters may comprise tools for dynamically 
creating sets that are the result of user defined criteria for picking stocks including 
security type, country, market cap, P/E, analysts following, and earnings growth. 
Fig. 14 depicts an embodiment of a screen that enables a user to utilize these tools. 

A model managing tool may also be provided and screens may be provided to 
enable users to manage these models. Such screens may comprise those depicted in 
Figs. 15-24. A backtest screen used to apply a model to historical data to test the 
model's accuracy may be provided, such as depicted in Fig. 25. 

Also, as shown in Figs. 26-27, historical view screens may be provided. This 
provides a visual depiction of analyst's estimates over time and may be used to view 
trends, cluster points, visually backtesting models by placing a smart model that is 
calculated every N days in the chart as a visual analyst to visually see its performance 
against the consensus and the other analysts. 

Also, as shown in Fig. 28, an analysis screen may be provided that presents 
the impact on the change in estimates and the change in stock prices or consensus 
estimates. The user may select a time frame to measure these factors, a time bin to 
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analyze the appropriate data sources and a set of stocks to analyze. The application 
then calculates the answers and presents a chart, such as the one depicted in Fig. 28. 

A source selector screen may also be provided to enable the user to select 
sources for performing this analysis. In the analysis, the variables may include the 
5 period, start date, price change/consensus change, source, source driving condition, 
and stock/stock sets to analyze and the sources may include brokers, analysts, 
clusters, and smart estimates. 

A performance module may also be provides that enables users to see past 
performance at the broker, analyst and ticker level. The user may select the prior 
10 periods, performance type and error type to view. 

Other views may also be presented as would be apparent to one of ordinary 
skill in the art and other embodiments and uses of the invention will be apparent to 
those skilled in the art from consideration of the specification and practice of the 
invention disclosed herein. Accordingly, the specification and examples set forth 
15 above should be considered exemplary only. The scope of the invention is only 
limited by the claims appended hereto. 
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CLAIMS 

What is claimed is: 

1 . A computer implemented system for facilitating use of historical data 
for a plurality of analysts, the system comprising: 

(i) a database of raw historical data, said data including individual 
predictions by a plurality of analysts pertaining to at least one security; 

(ii) means for creating indices for predetermined relationships 

among the data; 

(iii) means for pre-calculating predetermined analyst-performance 
metrics from the data; and 

(iv) database means for storing the indices and pre-calculations. 

2. The system of Claim 1 wherein the historical data comprises analysts' 
estimates of future earnings and the indices include an earliness bin index. 

3. The system of Claim 1 wherein the historical data comprises analysts' 
estimates of future earnings and the pre-calculated performance metrics comprise one 
or more of a raw error metric, an absolute error metric, a relative error metric, a bias 
error metric, a leadlag factor and a hit percent. 

4. The system of Claim 1 wherein the historical data comprises analysts' 
estimates of future earnings and when the estimate was made and the pre-calculated 
performance metrics comprises one or more of a raw error metric, an absolute error 
metric, a relative error metric, a bias error metric, a leadlag factor and a hit percent. 
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5. A computer implemented system for displaying historical data for a 
plurality of analysts, the system comprising: 

(i) a database comprising historical data, said historical data 
including individual predictions by a plurality of analysts pertaining to at least one 

5 security; 

(ii) a user interface comprising a plurality of user selectable 
parameters to permit a user to selectively display portions of the historical data; 

(iii) a display for displaying historical data according to the user 
selected parameters. 

10 6. The system of claim 5 wherein the parameters include one or more of 

a security, an event type, event dates, earliness bins for the predictions, and analysts. 

7. The system of claim 5 wherein the predictions comprise earnings 
estimates and the user interface further enables a user to selectively display one or 
more of: 

15 (i) actual historical earnings for a selected security; 

(ii) high historical estimates; 

(iii) low historical estimates; and 

(iv) consensus historical estimates. 

8. The system of claim 5 wherein the predictions comprise earnings 
20 estimates and wherein the historical data comprises analysts' estimates of future 

earnings and any revisions to these earnings estimates and when the estimates or 
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revisions were made, and the user can cause the display to selectively display a time 
series view of a selected analyst's estimates and revisions for a selected security. 

9. The system of claim 5 wherein the predictions comprise earnings 
estimates and wherein the historical data comprises analysts' estimates of future 
earnings and revisions to those earnings estimates and when the estimates or 
revisions were made, and wherein the user can cause the display to selectively 
display simultaneously a time series view of a selected group of analysts' estimates 
and revisions by analyst. 

10. The system of claim 5 wherein the predictions comprise earnings 
estimates for securities and wherein the historical data comprises analysts' estimates 
of future earnings and any revisions to those earnings estimates and when the 
estimates or revisions were made, and wherein the user interface further enables a 
user to selectively display simultaneously for a selected security a time series view of 
a selected group of analysts' estimates and revisions by analyst and one or more of: 

(i) actual historical earnings for the selected security; 

(ii) high historical estimates; 

(iii) low historical estimates; and 

(iv) consensus historical estimates for that security. 

1 1 . The system of claim 5 wherein the predictions comprise earnings 
estimates and wherein the historical data comprises analyst's estimates of future 
earnings and any revisions to those earnings estimates and when the estimates or 
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revisions were made, and the user can cause the display to selectively display 
simultaneously a time series view of a selected group of analysts' estimates and 
revisions by analyst for a selected time period prior to the earnings event. 

12. A computer implemented system for measuring historical 

5 performance of earnings estimates for a plurality of analysts, the system comprising: 

(i) a database of historical data, said data including individual 
earnings estimates by a plurality of analysts pertaining to at least one earnings event 
for a given security and actual earnings data for that event for that security; 

(ii) means for measuring selected analysts' performance based on 
1 0 predetermined error metrics; and 

(iii) and means for graphically displaying the results of the 

measurements. 

13. The system of claim 12 wherein the historical data comprises one or 
more of: an analyst identifier, an event identifier, an estimate date, and actual 

15 earnings for the event. 

14. The system of claim 12 where the error metric comprises one or more 
of a raw error metric; a relative error metric, a bias error metric, and a user-defined 
error metric. 

1 5 . The system of claim 1 2 further comprising: 
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(i) a user interface comprising a plurality of user selectable 
parameters to permit a user to selectively choose parameters for use in comparing 
analysts performance; and 

(ii) a display for comparatively displaying analysts performance 
based on the user selected parameters. 

16. The system of claim 12 wherein the parameters comprise one or more 
of a ticker, an event type, one or more event dates, a recency period relative to an 
event, one or more analysts and an error metric. 

17. The system of claim 12 wherein the parameter comprises an earliness 
bin, said earliness bin determining which data will be used in calculating error metric 
values. 

18. A computer implemented system for creating a composite prediction 
based on a plurality of analysts' current predictions and historical data concerning 
analysts' past predictions, the system comprising: 

(i) a historical database of data concerning a plurality of analysts' 
past predictions; 

(ii) a database of a plurality of analysts' current predictions; and 

(iii) means for creating a model with user selected parameters to 
generate a composite prediction based on the historical data. 

1 9. The system of claim 1 8 wherein the prediction comprises an analyst's 
earnings estimate for a security. 
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20. The system of claim 19 wherein the means for creating a model 
comprises means for modifying analysts' current estimate based on the analysts 
historical accuracy. 

2 1 . The system of claim 1 9 wherein: 

5 (i) the means for creating a model comprises means for applying 

a weighting factor; and 

(ii) a composite earnings estimate is generated based on the 
current earnings estimates modified by the weighting factor. 

22. The system of claim 1 9 wherein: 

1 0 (i) the means for creating a model comprises means for enabling a 

separate adjustment factor to be applied to current earnings estimates for selected 
analysts and a separate weighting factor to be given to the adjusted current earnings 
estimate; and 

(ii) a composite earnings estimate is generated based on the 
1 5 weighting factor of adjusted current earnings estimates. 

23. The system of claim 1 9 further comprising means for backtesting a 

model. 

24. The system of claim 19 further comprising means for applying a 
model to current estimates. 

20 25. The system of claim 1 9 further comprising means for updating the 

database with new estimates and re-applying the model to the revised estimates. 
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26. The system of claim 24 further comprising means for screening stocks 
whose composite estimate varies from consensus estimate by predetermined criteria. 

27. The system of claim 1 8 wherein the prediction comprises analysts' 
buy-seli recommendations for a security. 

28. The system of claim 27 wherein: 

(i) the means for creating a model comprises means for applying 
a weighting factor; and 

(ii) a composite buy-sell estimate is generated based on the current 
buy-sell recommendations modified by the weighting factor. 

29. The system of claim 27 wherein: 

(i) the means for creating a model comprises means for enabling a 
separate adjustment factor to be applied to current buy-sell recommendation for 
selected analysts and a separate weighting factor to be given to the adjusted current 
buy-sell recommendation; and 

(ii) a composite buy-sell recommendation is generated based on 
the weighting factor of adjusted current buy-sell recommendation. 

30. The system of claim 29 wherein the adjustment factor is based on a 
raw error metric; the weighting factor is based on an absolute error metric; and a 
composite buy-sell recommendation is generated based on the weighting factor 
weighted average of adjusted current buy-sell recommendation. 
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31. A method for assisting in an investment decision, the method 
comprising the steps of: 

(i) analyzing historical data pertaining to a plurality of analysts' 
historical predictions concerning a security; 
5 (ii) identifying trends within the data; 

(iii) modifying analysts current predictions based on the identified 

trends; 

(iv) creating a composite prediction based on the modified 

prediction. 

10 32. The method of Claim 3 1 wherein the prediction is an estimated 

earnings prediction. 

33. The method of Claim 3 1 wherein the prediction is a buy-sell 
recommendation. 
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Figure 1 1 b 
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Figure 1 1 c 
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